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ABSTRACT 

Statewide minimum competency testing programs have 
emphasized basic skills in reading, mathematics, and writing. 
However, continuing concerns are expressed in national reports about 
the level of achievement, particularly in mathematics and science, 
and increased testing has been suggested as a means of encouraging 
curriculum change and evaluating student progress, increased testing 
in current competency testing programs is unlikely to meet these 
broader goals. Two frameworks have implications for evaluating 
competency testing: (1) the context of assessment— testing of 
cognitive knowledge, learner characteristics which have an impact on 
instruction, and curricular validity; and (2) the meaningfulness of 
test scores and reports to students for self-direction, and to 
teachers for instructional planning. These frameworks can further 
increase the links between instruction and the assessments of 
individual students. A three-page list of references concludes the 
document. (Author/GDC) 
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Statewide competency testing programs .have emphasized basic 
skills In reading, mathematics, and writing. However, continuing 
concerns are expressed in national reports about the level of ach- 
ievement, particularly in mathematics and science, and Increased 
testing has been suggested as a means of encouraging curriculum 
change and evaluating student progress. Increased testing in 
current competency testing programs is unlikely to meet these 
broader goals. Two frameworks are examined for their Implications 
for evaluating competency testing: 1) the context of assessment 
(for competency scores), and 2) the meaningful ness of test scores/ 
reports to students for self-direction and to teachers for Instruc- 
tional planning. These frameworks can further Increase the links 
between Instruction and the assessments of Individual students. 
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Next steps in competency testing programs: 
Assessment in context and the mean ingfulness of results 

Carol Kehr Tittle 
Queens College of the ^ 
Cfty University of New York 

The rapid and continuing growth of large scale testing and assess- 
ment programs has historical rnntQ in staridardlzed testing programs. 
Although many different purposes are stated for competency testing 
programs, e. g., certification and setting standards for students, 
accountability and educational policy, and curriculum and educational 
program evaluation, they are not new purposes. In many aspects current 
programs provide little change In the types of tests and skills emphasized 
In past standardized testing programs. Yet there are changes In our 
views of the learner and the teaching process that will lead to changes 
In competency testing programs. Part of these changing views are based 
on a social cognitive perspective, to use Bandura's (1986) terms, on 
human motivation, thought and action. 

The directions of these changes are Indicated In a special issue 

of Educational Measurement (Summer, 1985). In several states, such as 

Connecticut, with NAEP item administrations, proficiency testing in 

grade 9, mastery testing of higher order academic skills (grades i»,6, S 8), 

and the use of fall testing programs, testing has been more concerned with 

the curricular and instructional impact of the testing program. Testing 

'paper presented at the annual meeting of the American Educational 
Research Association, San Francisco, CA, April 1986. 
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is also seen as more than basic skills assessment (and see- Ihe 
extension to other subject areas proposed in Florida). 

In the same issue, however, Guthrie and Lissitz point out the 
importance of distinctions among the £S£ of tests, and Elliott and 
Hall call attention to ''new'* purposes of tests and measures. These 
purposes include: Dwhether the broader assessment programs include 
measures known (or likely) to be related to student performance; 
?) whether measures include those that educators or policy makers can 
change; and 3) whether the measures are understandable. These are three 
useful criteria and I will apply them from a different perspective for 
the state assessment programs that test all students. This perspective 
is the concern with classroom instruction. 

Because competency testing has in many instances demonstrated Its 
effectiveness or leverage on schools, teacliers, and students, there is 
now a need to re-examine the general perspectives or assumptions on 
which we develop and evaluate statewide assessment programs. There is 
a need to go beyond defining objectives and setting standards as these 
programs expand the subject areas and grades tested. For this re- 
examination we need a framework that permits an understanding of test 
results on the basis of the context in which they are embedded. We 
also need a framework that permits an understanding of how meaningful 
test scores and reports are to students and other decision makers-- 
teachers, parents, curriculum planners, and policy makers, fn the next 
sections two tasks are undertaken: 

1, The examination of the context of scores and the implications of 
this view for assessment programs; and 

2, The examination of the meaningfulness of test scores/reports, to 
students and instructional planners, and the implications of this 
view for state-wide testing programs. 



3 

The focus on context ts to enhance instructional planning, building 
from a research base to identify modifiable variables that relate to 
the achievement performance of students. The focus on mean ingful ness 
of test scores/reports is to enhance instructional planning and student 
self-direction, building on studies of student uses of evaluative information 
and studies of teacher decision processes and uses of test results. 

Assessment in Context 
Messick (1984) has summarized a broad perspective for the measurement 
process. The framework implies, "...a strategy of comprehensive 
assessment in context that focuses on the processes and structures involved 
in subject matter competence as moderated in performance by personal 
and environmental influences' (.p. 215). The three categories of the 
framework are: 1. subject matter competence; 2. intraindi vidual (student) 
characteristics; and 3. situational contexts. To understand assessment 
results in context means that 

1) the acquisition of knowledge and cognitive skills are seen in 

developmental terms. One aspect of this developmental perspective is 
that tKe constructs emphasized in educational measures may differ 
according to wKether students are at the beginning, intermediate or 
advanced levels of learning of the subject area. Eventually we may 
he able to construct items and tasks that relate the sources of 
student difficulties to the cognitive process and structures that 
appear at different developmental levels. Attempts to measure 
problem solving skills appear to incorporate some aspects of this 
perspective, and draw on research on expert and novice approaches to 
probilem solving. 
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I am less optimistic than Messick that the use of independent assessments 
of cognitive abilities (independent of achievement or knowledge-based 
problems of achievement) will be useful in instructional planning. Such 
measures seem an extension of the psychological trait approach and instruc- 
tion remains embedded in subject matter; Burstall (I986) may provide an 
alternative approach, as do the approaches of Fuerstein and others. 

Present research, however, provides support for more immediate use 
of the other two components of assessment in context. These two arise 
Initially from the need to rule out alternative interpretations of 
performances, particularly poor performance. More directly, these two 
components have valuable contributions to make to instructional planning. 

2) student characteristics may suggest facilitating or Interfering 
Influences on achievement. Particularly of concern here are 
affective and motivational characteristics that may provide direction 
to Instructional planning. (In Messlck's (.1984) discussion, drawing 
on the special education area In which the framework was developed, 
there Is emphasis on screening for biomedical factors and emotional 

or behavioral deviance that would be destructive of classroom Instruc- 
tion and test performance.) 

3) situational contexts also Influence test performance. Of particular 
concern here Is whether the student has bad an opportunity to learn 
the knowledges and skills assessed on the test. While other aspects 
of the student's soclocul tural environment may be of Interest, such 

a modlflaBle variable Is directly related to Instructional planning. 

The Implications of the last two components for competency testing 
programs are supported by studies In the research and evaluation literature. 
For the situational context, Messick, for exampU, raises the question 
whether a student needs remediation, a different mode of Instruction, or 

6 
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better teaching withfn the c>Trent mode. Some evidence related to these 
Instructional planning decisions can be developed by examining the match 
of curriculum (texts, materials), instruction, and test items for individual 
students. Studies which have examined aspects of opportunity to learn 
suggest that as states move to testing other curriculum areas it may be 
important (at least for samples of classrooms) to have teachers indicate 
which items in tests students in fact have had an opportunity to learn in 
texts and in instruction, CSee, for example, Porter et a1, no date; Cooley 
6 Leinhardt, i^SO; Leinhardt & Seewalu, 1301 i Haii:>yii, Kunorrib, 
Bailey, 1386.) 

For the student characteristics component (.2), an implication is to 
include measures of student affect or motivation that appear to be 
associated with cognitive learning Ce,g,, Messick, 1979; 1985), As an 
illustration, consider the current status of research on gender and 
mathematics. There have been a series of studies over the past 15 years, 
including those by Fennema and her colleagues (Pennema S Sherman, 1977; 
Fennema, Wolleat, Pedro S Becker, 1981; Fennema S Peterson, 1985), by 
Eccles and her colleagues (.1983, 198^, 1985), as well as many other re- 
searchers Csummarized in ChTpman, Brush S Wilson, 1985; Dweck, 1986; and 
Tittle, 1986), This research has yielded findings to date that have 
Implications for the assessment in mathematics of characteristics of 
both hoys and girls. Among these findings are: 

the importance of student perceptions as influences in mathematics 
achievement and course taking (participation in further and advanced 
mathematics), particularly as reflected in measures of 

. interests, especially career-related interests, and the perceived 
need for mathematics and its usefulness 
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• attitudes toward mathematics, towards onesel f as a learner in 
mathematics and one's expectations for and attributions for 
success and failure, in perceptions of task difficulties 

• positive support from others such as teachers and counselors. 
• the general decrease in positive attitudes towards mathematics as 

students move from the elementary education level to the secondary level 
Student attitudes, interests, expectancies and values are linked with 
educational achievement, grades and continuing participation in mathematics. 

Resu^la uT uiassrcom observatTon studies suggest that teachers 
typically do not include statements or discussions that are designed to 
influence these perceptions. (See the studies reported in Wilkinson & 
Marrett, I985, and Stodol sky's descriptions of elementary school class- 
rooms. Telling math: Origins of math aversion and anxiety, I985.) 
Thus an extension of student measures to the affective area offers 
opportunities to link such assessments to reports of achievement measures. 
This may increase the usefulness of test results to students and to 
teachers in planning instruction. Teachers' statements and activities 
that might influence student motivation (causal attributions, confidence, 
perceived usefulness of mathematics) are now identified at a level of 
specificity that is likely to add to the meaningfulness of reports to 
teachers and students. Research is continuing to examine constructs 
such as autonomous learning behaviors (Grieb S Easley, 1984; Fennema & 
Peterson, I985) and adaptive motivational patterns (.Dweck, I986). 

The tmplications of this research appear to be translatable into 
assessment measures that can accompany current competency achievement 
measures. Further implications are found in th.e re-examination of how 
test scores are used and how results are reported to students, teachers 
and parents. 
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Meaningful ness of Test Scores and Results 

There Js a small and growing body of research on teacher use of 

test results in Instructional planning and reports that teachers might 

prefer for Instructional planning purposes (Rudman et al, 1980; 

Shavelson & Stern, I98I; DeCasper, I985), as well as surveys of teacher 

attitudes toward minimum competency testing (described in Tittle, I982). 

A related area of research examines the meanlnp that Individuals (such as 

studcr.ts) dcilve from tesst snores or other evaluative Inrurmatlon. The 

importance of this research rests on the view of the learner, mentioned 

earlier. The self-regulation of motivation and one's actions operate 

partly through Internal standards and evaluative reactions to one's own 

performances (Bandura, I986). In Bandura's view. 

Among the types of thoughts that affect action, none is more central or 

pervasive than people's judgments of their capabilities to deal 

effectively with different realities. It Is partly on the basis of 

self percepts of efficacy that they choose what to do, how much 

effort to invest In activities, how long to persevere In the face of 

disappointing results, and whether tasks are approached anxiously or 

sel f-assuredly. . /'(p. 21) 

Drawing on these research areas, several criteria may be approrlate 
for examining the meanlngfulness of test results to students and to 
teachers. These are Identified separately for students and teachers. 
For students: 

1) The (.literal) comprehension of test scores and Interpretive reports. 
What Is the accuracy with which terms such as percent, percentile, 
standard scores, above/below average, are understood by students? 
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2) The perception of test informattm 
Students interpret events (test results) and organize the Information 
derived from them into beliefs about what leads :o what. In Bandura*s 
terms, test scores are environmental cues, affecting the likelihood of 
particular actions, such as deciding to stay in school, deciding; to work 
harder, through their judged predictive function. 

At least two areas are pertinent here. The first area is subject 
matter learning. 

What does the student understand to be the meaning of the test 
information for further academic progress? For learning activities 
to be undertaken? For predicting teacher actions? For predicting 
parent actions? 

The second area is that of self-direction (self-efficacy, self-regulation). 
What does the student take as the meaning of the information for 
the future? For example, for expectations for continued levels of 
performance or persistence in the subject matter (values, attitudes, 
interests, occupational goals)? If the student decides to increase 
(or decrease) learning activities, what outcomes are these actions 
1 ikely to produce? 

Thus the criteria for the meaningfulness of test scores and 
reports to students are concerned with the accuracy of communication and 
with the socially constructed interpretations or meanings that result 
when students receive test scores. The importance of the criterion of 
the student's perceptions and use of th.e information to construct 
expectations is supported by recent work in motivational theory cited 
earlier (.Bandura, 1986; Dweck, 1986). 
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Several types of studies are suggested by these criteria. For 
example, studies may vary the interpretive material in which test scores 
are embedded for the effect on students with different motivational 
characteristics and performance levels. Studies may also examine the 
effect of teacher interpretations designed to affect self efficacy of 
perceptions of students with different motivational and achievement 
levels. Some guidance as to the variables to be examined in student 
perceptions is also given in Bandura (1986, p. 337). As part of the 
seif-moni torino prnrpss, test scores can be described as observations 
that can be examined for accuracy, appropriateness/relevance, and for the 
standards the individual uses in judging the information (.personal 
standards, referential performances, valuation of activity, and 
performance attribution—personal or external locus), and self-reaction 
(positive, negative, tangible, or no self-reaction). 

These criteria and perspective encourage the study of test information 
Itself for Its meaning to students as self-diagnostic and self- 
motivating Information. Similar criteria would apply to determine the 
meanlngfulness of test scores and results to parents. The focus would 
be on comprehension and perceptions for predicting their children's 
academic and motivational acttons In the future. 

The criteria for meanlngfulness need further elaboration for teachers. 
The criteria of comprehension and perceptions of test information are 
appropriate for understanding the test results \n relation to an 
Individual student. However, there are additional criteria that are 
related to the instructional process in two areas. Instructional planning 
and teacher self-efficacy. 
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For teachers ; 

1) The (literal) comprehension of test scores and interpretive 
reports. 

What IS the accuracy with which terms .such as percent, percentile, 
standard scores, above/below average, are understood by teachers? 

2) The perception of test information 

What does the teacher understand to be the meaning of the test 
information for further academic progress of the student? 
WiidL tiue^ Lilts L'-.dcIier ufiueidLdMu Lne iikely meaning of the ihiior* 
mation for the student's perception of further academic progress 
and self-direction (future actions)? 

3) The application of test Information to Instructional planning , 
Its meanlngfulness for instruction. 

What does the teacher (curriculum decision maker) plan to do 
In the Instructional process based on the test Information? 
Planning will incorporate both areas, student academic (subject 
matter) learning and self development (jnotlvatlon and affective) 
influences on performance. 
^) The application of test information for teacher self-efficacy . 
What does a teacher predict will be the outcomes of instructional 
planning based on tests scores/results? 

What effect does the test report have on/?eacher's perceptions of 
her/his ability to enhance student achievement/motivation? 
Test reports are being expanded in content and Interpretations are 
being provided to link test scores to Instructional planning. For 
example, statements of objectives and references to Instructional 
resources are used or are potentially available for reports . (DeCasper, I985), 
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BtcauM of the Important role of the Individual's perceptions of 
Information In guiding future actions, It Is suggested as another 
criterion for examining the meanlngfulness of test Information to 
teachers. 

Studies of teacher criteria can also take a variety of forms, as 
suggested by the studies for the student criteria. Although rot elaborated 
here, the criteria for eve luting competency assessment programs needs 
to Include school factors that affect the usefulness of results 
(fsctorc such zz the tljr.lr.u .riJ organization for reporting of Individual 
results to teachers, OeCasper, 1985). 

Summary 

''Validity Is the most Important consideration In test evaluation. 
The concept refers to the appropriateness, meanlngfulness, and usefulness 
of the specific Inferences made from test scores" ( Standards . I985, p. 9). 

In this discussion I have been concerned with aspects of the 
validity of competency testing programs. I am suggesting an evaluation 
of the link between testing and instruction, within a view that sees the 
learner and the teacher as responsible agents In the educational process. 
I have used two concepts, assessment In context and meanlngfulness , that 
need to be added to our ideas of content In developing validity- 
related evidence for educational measurements. . Assessment in context 
fits well within the traditional view of construct-related evidence of 
validity. Thus assessment In context Is a useful addition to the validity 
related eviden- for educational measurement. 

Validity-related evidence for the meanlngfulness and usefulness 
of specific inferences from test scores or interpretive reports have not 
been well specified; they are usually noted by their absence, without 
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criteria to define their presence. 

I have suggested that In educational measurement we need validity- 
related evidence that gives substance to the Ideas of appropriateness, 
meanlngfulness and usefulness of Inferences based on test scores. 
In order to do this we must go beyond content-reiated judgments to 
strengthen our links with other researchers In education and to draw 
on new frameworks for Integrating teaching, ledrnlng, and testing. 
The resources and ImpurLance of the competency Lebtlnq programs 
suggest the state education agencies will provide leadership In expanding 
our frameworks for evaluating assessment programs. 
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